
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



10/669,593 



09/25/2003 



ICazuo Shiota 



2292 7590 03/17/2005 

BIRCH STEWART KOLASCH & BIRCH 
PO BOX 747 

FALLS CHURCH, VA 22040-0747 



209 1-029 IP 



1320 



EXAMINER 



ART UNIT 



WANG, JIN CHENG 

1 



PAPER NUMBER 



2672 



DATE MAILED: 03/17/2005 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 


Application No. 

10/669,593 


Applicants) 
SHIOTAETAL 


Examiner 

Jin-Cheng Wang 


Art Unit 

2672 





-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 
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THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
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Application/Control Number: 10/669,593 p age 2 

Art Unit: 2672 

DETAILED ACTION 
Claim Rejections - 35 (JSC § 102 

(e) the invention was described in (1) an application for patent, published under section 1 22(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 2 1 (2) of such treaty in the English language. 

Claims 1, 3-5, 6, 8, 10, 1 1, 13, 15, 16, 18 and 20 are rejected under 35 U.S.C 102(e) as 
being anticipated by Loui et al. U.S. Patent No. 6,636,648 (hereinafter Loui). 

Re claims 1, 6, 1 1, and 16, Loui discloses a method for generating an album based on 
album data including at least one image data set, which has been photographed during a trip and 
which has location data representing a photography location attached thereto, comprising the 
steps of: .* 

Calculating a distance between the photography location of the image data set and a 
predetermined reference position, based on the location data (e.g., Loui teaches in column 9-11 
adding annotations for pictures and associating metadata to the pictures. Loui teaches an 
albuming algorithm that arranges the picture according to an event or a sub-event wherein the 
event or sub-event are defined by the metadata including metadata tags and GPS information to 
determine location and subject depth information. The information defines the context 
surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event. Loui classified the pictures using the metadata having the 
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location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries)', 

Judging whether the distance is over a predetermined threshold value {e.g., Loui teaches 
in column 8-11 adding annotations for pictures and associating metadata to the pictures. Loui 
teaches an albuming algorithm that arranges the picture according Jo an event or a sub-event 
wherein the event or sub-event are defined by the metadata including metadata tags and GPS 
information to determine location and subject depth information. The information defines the 
context surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event. Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries), 
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Classifying the image data set according to the result of judgment (e.g., Loui teaches in 

column 8-11 adding annotations for pictures and associating metadata to the pictures. Loui 

teaches an albuming algorithm that arranges the picture according to an event or a sub-event 

wherein the event or sub-event are defined by the metadata including metadata tags and GPS 

information to determine location and subject depth information. The information defines the 

context surrounding a picture capturing event allowing the user to select the set of pictures in the 

database using an arbitrary query based on the stored metadata and the selected set of pictures 

is filtered by omitting those pictures whose, image, quality metric is below some threshold and the 

event segmentation stage determines event and sub-event boundaries to determine final set of 

pictures for each event and sub-event Loui classified the pictures using the metadata having the 

location information and the pictures are classified in accordance to the predetermined way of 

* 

dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries, e.g., the distance is either zero or larger than a threshold value so that the photo is 
judged to be either within or outside the event and sub-event boundaries) , 

Generating the album data according to the result of classification (e.g., Loui teaches in 
column 7-11 the page layout module presents to the user an automatically laid out album 
organized by event with the algorithm choosing the albuming parameters by default or specified 
by the user). 
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Re claims 3-5, 8, 10, 13, 15, 18 and 20, Loui teaches in column 8-1 1 adding annotations 
for pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
arranges the picture according to an event or a sub-event wherein the event or sub-event are 
defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some-threshold and the event segmentation stage 
determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value forjudging the distance 
between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries, e.g., the distance is either zero or 
larger than a threshold value so that the photo is judged to be either within or outside the event 
and sub-event boundaries. Loui further teaches in column 7-1 1 the page layout module presents 
to the user an automatically laid out album organized by event with the algorithm choosing the 
albuming parameters by default or specified by the user. Therefore, Loui further discloses the 
claim limitation of "obtaining related data, which is related to the photography location of the at 
least one image data set for which the distance is over the predetermined threshold value, based 
on the location data attached thereto, from a related data storage means which stores a plurality 
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of related data sets" wherein the related data set is the metadata including the location of 
photographing stored in the database. Loui further discloses the claim limitation of "generating 
album data, which includes the related data" because Loui discloses displaying album organized 
by event wherein the event are associated with the pictures in the album. 

Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 2, 7, 9, 12, 14, 17, and 19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Loui et al. U.S. Patent No. 6,636,648 (hereinafter Loui), in view of Kino et al. 
U.S. Patent No. 6,832,101 (hereinafter Kino). 

Re claims 2, 7, 12, and 17, Loui teaches in column 8-1 1 adding annotations for pictures 
and associating metadata to the pictures. Loui teaches an albuming algorithm that arranges the 
picture according to an event or a sub-event wherein the event or sub-event are defined by the 
metadata including metadata tags and GPS information to determine location and subject depth 
information. The information defines the context surrounding a picture capturing event allowing 
the user to select the set of pictures in the database using an arbitrary query based on the stored 
metadata and the selected set of pictures is filtered by omitting those pictures whose image 
quality metric is below some threshold and the event segmentation stage determines event and 
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sub-event boundaries to determine final set of pictures for each event and sub-event. Loui 
classified the pictures using the metadata having the location information and the pictures are 
classified in accordance to the predetermined way of dealing with the actual boundary condition 
of events and sub-events. This classification algorithm according to the event and sub-event 
boundary involves a predetermined threshold value forjudging the distance between the photo 
location contained in the metadata and a reference position associated with an event or sub-event 
such as the event and sub-event boundaries, e.g., the distance is either zero or larger than a 
non-zero threshold value so that the photo is judged to be either within or outside the event and 
sub-event boundaries. Loui further teaches in column 7-1 1 the page layout module presents to 
the user an automatically laid out album organized by event chronologically or non- 
chronologically with the algorithm choosing the albuming parameters by default or specified by 
the user. The laid out album organized by events arranged in a chronological order requires the 
location information for the events and sub-events along the route. Therefore, Loui further 
discloses the claim limitation of "generating travel route data, which represents a route taken 
during the trip, based on the location data attached to the at least one image data set for which the 
distance is over the predetermined threshold value" because classifying the events and sub- 
events based on the location are travel route data which represents a route taken during a trip for 
locations of the pictures arranged in a chronological order. Loui further discloses generating 
photography data such as the picture data and the metadata which represents that the image data 
set was obtained at the photography locations along the route. 

Loui does not disclose "obtaining a map data set that contains the route from a map 
database which stores a plurality of map data sets, based on the travel route data" and similarly 
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"attaching the photography data to the map data set and generating album data including the map 
data set, in which the photography data is correlated with the image data set." 

However, Kino discloses a method of photographing a photo image at a construction side 
to be related with a map and registering it in the database wherein the information about the 
photograph position on the map is stored in the database to be related with attached information 
about the construction site. Kino further discloses associating a symbol to the place where the 
image was photographed on the map allowing for searching the image and the attached 
information to be related each other by referring to the symbol list using the symbol ID. Kino 
thus discloses the locations of the travel route data chronologically because the locations of the 
pictures taken are ordered chronologically along a travel route. In this way, the locations are 
searched/extracted from the database based on the attached information and the locations thus 
extracted in a chronological order form a travel route (See Kino column 3, lines 50 through 
column 4, lines 16). 

Although Loui did not specifically teach the claim limitation, it is well known to one of 
the ordinary skill in the art to have incorporated Kino's map database to have associated symbols 
or metadata's locations to the photographs and to have determined a travel route based on the 
Loui's location data associated with the pictures. 

One having the ordinary skill in the art would have been motivated to do this because it 
would have provided a travel route on the map database and means for relating the position on 
the map with the symbols attached to or associated with the pictures (See Kino column 3, lines 
50 through column 4, lines 16). 
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Re claims 9, 14 and 19, Loui teaches in column 8-1 1 adding annotations for pictures and 
associating metadata to the pictures. Loui teaches an albuming algorithm that arranges the 
picture according to an event or a sub-event wherein the event or sub-event are defined by the 
metadata including metadata tags and GPS information to determine location and subject depth 
information. The information defines the context surrounding a picture capturing event allowing 
the user to select the set of pictures in the database using an arbitrary query based on the stored 
metadata and the selected set of pictures is filtered by omitting those pictures whose image 
quality metric is below some threshold and the event segmentation stage determines event and 
sub-event boundaries to determine final set of pictures for each event and sub-event. Loui 
classified the pictures using the metadata having the location information and the pictures are 
classified in accordance to the predetermined way of dealing with the actual boundary condition 
of events and sub-events. This classification algorithm according to the event and sub-event 
boundary involves a predetermined threshold value forjudging the distance between the photo 
location contained in the metadata and a reference position associated with an event or sub-event 
such as the event and sub-event boundaries, e.g., the distance is either zero or larger than a 
threshold value so that the photo is judged to be either within or outside the event and sub-event 
boundaries. Loui further teaches in column 7-1 1 the page layout module presents to the user an 
automatically laid out album organized by event with the algorithm choosing the albuming 
parameters by default or specified by the user. Therefore, Loui further discloses the claim 
limitation of "obtaining related data, which is related to the photography location of the at least 
one image data set for which the distance is over the predetermined threshold value, based on the 
location data attached thereto, from a related data storage means which stores a plurality of 
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related data sets" wherein the related data set is the metadata including the location of 
photographing stored in the database. Loui further discloses the claim limitation of "generating 
album data, which includes the related data" because Loui discloses displaying album organized 
by event wherein the event are associated with the pictures in the album. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (703) 605-1213. 
The examiner can normally be reached on 8:00 - 6:30 (Mon-Thu). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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